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NB  -  although  awarded  in  year-by-year  amounts,  this  was  a  four-year  grant  with  grant  letters  specific  to 
year  1  and  then  to  year  2  containing  options  for  re-funding  for  years  3  and  4. 

Preamble  to  Final  Report:  This  was  a  very  successful  body  of  work  performed  in  close  collaborations 
with  three  other  laboratories  in  the  UK  and  USA.  It  resulted  in  several  exciting  discoveries,  over  70  peer 
reviewed  publications  (39  with  direct  contribution  from  Australian  section),  several  in  Science,  Nature 
and  other  leading  scientific  journals  and  between  us  close  to  70  conference  presentations.  This 
combination  of  two  biologists  a  physicist  and  an  engineer  has  worked  very  effectively  and  it  is  hard  to 
ascribe  particular  publications  or  discoveries  to  any  one  laboratory.  Nonetheless  in  what  follows  I  have 
tried  to  clearly  disambiguate  publications  and  discoveries  principally  led  by  my  lab.  and  which  therefore 
might  be  attributed  to  the  AOARD  section  of  this  overall  project. 

Abstract:  Stomatopod  (mantis  shrimp)  vision  is  both  unique  among  animals  and  extraordinarily 
complex  at  the  receptor  level,  comprising  a  total  of  20  different  photoreceptor  types  or  functional  input 
channels.  12  channels  for  color  (including  several  in  the  UV,  the  whole  system  sampling  from 
300-720nm),  6  for  linear  polarization  (4  sampling  at  0°,  90°, 45°  and  135°)  with  peak  spectral  sensitivity 
close  to  500nm  and  2  in  the  UV  sampling  at  0°  and  90°,  with  peak  spectral  sensitivity  close  to  350nm) 
and  (in  some  species),  2  for  circular  polarization  (Left  and  Right-handed),  also  with  peak  spectral 
sensitivity  close  to  500nm.  Why  do  stomatopods  sample  the  light  that  is  available  to  them  in  such  great 
detail? 

This  research  project  built  on  our  previous  work  in  this  system  and  with  a  combination  of  our  current 
state  of  knowledge  and  fresh  intellectual  and  methodological  input  to  the  project,  we  aimed  to  explain  the 
complexity  of  the  system  for  the  first  time.  Stomatopods  brains  are  small  and  are  unlikely  to  deal  in 
multi-dimensional  data  sets  but,  in  common  with  most  invertebrates,  more  likely  send  a  set  of  simple 
‘command  messages’  to  the  brain  from  the  eye  and  outer  visual  system  neuropils.  The  mechanisms  that 
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function  to  reduce  and  analyse  the  complexity  of  20  data  streams  make  up  an  important  component  of  the 
research  we  propose  here.  Added  leverage  to  approach  both  receptor  function  and  data  streaming  / 
filtering  came  from  new  collaborations  with  the  Gruev  Laboratory  that  can  mimic  or  re-engineer  the 
stomatopod  eye  optoelectronically.  By  sharing  coding  principles  between  disciplines  (engineering  and 
biology),  and  through  continued  behavioural,  anatomical,  molecular  and  physiological  investigations,  we 
aimed  to  decode  the  inner  principles  of  stomatopod  vision  as  well  as  provide  spin-offs  to  inform  more 
efficient  and  smart  sensor  design. 


Introduction: 

Over  the  course  of  many  years,  our  collaborative  work  on  the  complexity  of  the  stomatopod  (mantis 
shrimp)  visual  system  has  provided  many  new  discoveries  and  exciting  research  outcomes  (for  example, 
ten  papers  to  date  in  Nature/Science/Current  Biology).  In  terms  of  the  number  of  input  channels  at  the 
retinal  level,  their  vision  is  far  more  complex  than  any  other  vertebrate  or  invertebrate  system  known. 
Anatomically,  there  are  16  different  photoreceptor  types,  arranged  into  12  channels  for  color  (including 
several  in  the  UV,  the  whole  system  sampling  from  300-720nm),  6  for  linear  polarization  (4  sampling  at 
0°,  90°, 45°  and  135°)  with  peak  spectral  sensitivity  close  to  500nm  and  2  in  the  UV  sampling  at  0°  and 
90°,  with  peak  spectral  sensitivity  close  to  350nm)  and  (in  some  species),  2  for  circular  polarization  (Left 
and  Right-handed),  also  with  peak  spectral  sensitivity  close  to  500nm.  Altogether,  this  produces  a 
20-channel  sensor  that  has  the  potential  to  encode  most  or  all  spectral  and  polarization  information 
available  from  300-720nm. 

Our  overarching  aim  was  to  answer  two  integrated  questions: 

(1)  Why  these  parallel  streams  of  information  are  optimal  for  what  stomatopods  are  seeing  in  their  visual 
environment  and  the  complex  visual  tasks  stomatopods  perform. 

(2)  How  their  processors  (both  in  the  retina  and  in  the  central  nervous  system)  are  optimized  to  avoid  the 
seemingly  inevitable  log-jam  of  information  resulting  from  such  massively  parallel  input. 

A  key  element  in  their  visual  system  is  the  line-scanning  (push-broom)  sensor  array  used  by  stomatopods. 
This  multi-channel  line-scan  array,  thought  to  be  unique  in  animal  vision,  avoids  the  problem  of 
integrating  20  two-dimensional  (i.e.  spatially  extended)  arrays  of  information  and  extracting  salient 
information  from  this  stack  in  real  time.  It  presents  other  problems  however,  as  the  line-scan  system  is 
embedded  within  a  mostly  conventional,  extended  array  of  visual  sensors.  Thus,  the  overall  system 
requires  an  internal,  temporal  and  coordinate  system  on  which  visual  information  can  be  painted  during 
each  scan.  How  this  data-management  centre  functions,  or  even  where  it  is  located,  has  yet  to  be 
discovered. 

The  stomatopod  lifestyle  is  fast  and  full  of  potentially  deadly  combat  (Fig.  1).  In  fact,  their  raptorial 
strike  is  known  to  cause  cavitation  (in  essence,  boiling)  the  water  in  front  of  its  impact  point  as  they 
defend,  attack  and  find  food.  Behaviorally,  they  have  been  described  as  ‘primate- like’  in  their  complexity, 
and  it  is  almost  certainly  the  interactions  between  their  visual  system  and  their  violent  capabilities  that 
have  pushed  these  creatures  along  their  unique  evolutionary  trajectory  for  the  past  400  million  years.  We 
want  to  learn  from  this  evolutionarily  based  design.  As  their  closest  existing,  artificial 
design-neighbours  for  data  throughput  and  processing  complexity  are  airborne  and  satellite  sensors,  there 
are  already  engineering-biology  parallels  available  for  exploitation. 
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Fig.  1  Stomatopods  process  20  streams  of  visual  information  perhaps  faster  than  any  animal  known.  Here  Lysiosquilla 
maculata  catches  a  fish  in  a  few  thousandths  of  a  second.  (Photo  Roy  Caldwell). 


Invertebrate  sensory  systems  are  elegant  in  the  means  used  to  filter  out  and  reject  unwanted  information 
input  at  peripheral  (e.g.  photoreceptor  and  post-receptor  interneuron)  stages.  In  contrast,  vertebrate 
systems  typically  perform  much  of  the  filtering  in  the  brain.  The  result  for  humans  and  many  other 
vertebrates  is  a  huge  central  nervous  system  to  perform  this  relatively  time-consuming  and  energetically 
expensive  task.  In  engineering  machine  vision  systems,  we  have  often  attempted  to  emulate  the 
vertebrate  way.  As  our  sensors  have  become  more  refined  and  high-density  in  recent  years,  this  has 
resulted  in  processing  problems  that  prevent  fast  handling  of  incoming  information.  An  important 
advance  promised  by  the  research  described  in  this  proposal  is  that  by  examining  the  information-coding 
strategies  of  stomatopods,  it  could  become  possible  to  emulate  their  parsimonious,  energetically  and 
materially  efficient,  design.  Shrimp-brains  are  small  -  so  the  incoming  messages  for  survival  are 
necessarily  kept  simple. 

Our  previous  work  on  stomatopod  vision  has  taught  us  much  about  its  unique  visual  inputs  and  retinal 
design.  The  research  has  revealed  unexpected  dimensions  of  vision  such  as  circular  polarization 
sensitivity,  which  gives  these  species  a  covert  communication  channel  and  possibly  unique  ways  to  solve 
problems  of  scatter  and  object  detection  under  water.  There  have  always  been  unanswered,  important 
questions,  however.  Why  is  this  system  so  different  from  any  other?  What  visual  information  has  the 
system  evolved  to  see?  Why  use  12  color  channels  when  theoretically  only  6  are  needed?  Humans 
use  higher  order  polarization  analyses  such  as  Muller  Matrices.  With  all  the  available  information 
required  to  calculate  Stokes  parameters  (SO,  SI,  S2,  S3),  is  the  stomatopod  eye  also  capable  of  complex 
polarization  analysis? 
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The  specific  aims  the  project  attempted  are  listed  below  and  under  each  details  of  the  most  important 
outcomes  and  discoveries  are  given.  Following  this  there  is  an  overview  of  the  importance  of  the  work. 

(1)  To  learn  the  design  foundations  stomatopods  use  for  optimized,  rapid  processing  of 
multi-channel  information.  We  examined  how  polarization  is  viewed  by  stomatopods,  discovering  a 
fundamental  explanation  for  why  they  rotate  their  eyes  (Daly  et  al  2016,  Marshall  et  al  2014).  We  also 
made  the  first  discovery  of  a  behavioural  use  for  circular  polarization  vision  in  intra- specific  recognition 
(Gagnon  et  al  2015).  This  led  to  a  suggested  modification  of  the  way  polarisation  information  is 
displayed  for  humans  to  understand  in  any  imaging  polarisation  application.  Instead  of  colour  coding, 
intuitive  polarisation  shapes  are  used  to  express  degree,  angle  and  elipticity  (Gagnon,  Marshall  2016). 
Furthermore,  we  have  discovered  a  unique  type  of  UV  modulating  filters  to  provide  mutli-channel 
information  from  a  limited  palette  of  UV  visual  pigments  (Bok  et  al  2014,  Cronin  et  al  2014).  In 
comparative  work  the  relative  signalling  value  of  information  at  either  ends  of  the  spectrum  underwater 
were  considered  (Franklin  et  al  2016,  Wilkins  et  al  2016,  Stieb  et  al  2017).  Speed  and  acuity  of 
polarisation  information  processing  was  examined  behaviourally  with  the  surprising  finding  that  linear 
polarisation  cues  are  examined  in  as  much  detail  as  colour  or  luminous  (brightness)  information,  but  only 
in  some  animals  (Temple  et  al  2012,  How  et  al  2012,  How  et  al  2015). 

(2)  To  discover  how  information  is  processed  in  stomatopod  visual  systems. 

We  have  provided  evidence  for  a  new  type  of  multi-channel  color  processing  in  animals  (Thoen  et  al 
2014,  Zaidi  et  al  2014).  We  have  shown  how  stomatopod  processing  of  polarization  information  is 
optimized  not  to  suffer  confusion  points  as  are  seen  in  crabs  and  cephalopods  (How  2014)  and  erected 
the  idea  of  a  formalised  polarisation  distance  in  euclidial  space  -  just  as  colour  has  such  a  distance  in 
colour  vision  (How  and  Marshall  2014).  We  have  examined  the  molecular  diversity  in  the  retinas  of  these 
animals,  which  will  produce  a  much  better  understanding  of  the  development  of  polarization  vision  and 
of  the  roles  of  polarization  opsins  in  larval  and  adult  eyes.  Finally,  we  have  shown  that  multiple  opsins 
are  expressed  in  polarization  receptors  in  stomatopods  along  with  evidence  for  the  presence  of  a  “dorsal 
rim”  -  a  specialized  region  of  the  retina  that  in  insects  is  devoted  to  polarization  vision.  In  comparative 
work,  the  perception  of  illusions  was  examined  in  the  reef  environment  lending  support  to  the  idea  of 
processing  apparently  complex  tasks  early  on  in  the  visual  neural  train  (Simpson  et  al  2016).  Finally,  in  a 
series  of  papers  emerging  now,  we  are  beginning  to  describe  the  neuroanatomy  of  stomatopod  optic 
neuropils  and  brain  in  detail  (Thoen  et  al  2017,  Wolf  et  al  2017). 

(3)  To  discover  the  natural  complexity  of  visual  scenes  the  stomatopod  imaging  system  has  evolved 
to  see.  The  development  of  the  polarization  cameras  has  been  a  huge  success  (Powell  et  al  2014).  Not 
only  have  we  been  able  to  access  a  remarkable  amount  of  new  information  about  the  visual  ecology  of 
polarized  light  underwater,  we  have  been  able  to  translate  this  biomimetic  vision  into  fundamental 
medical  research  and  the  field  of  GPS  denied  navigation.  This  culminated  in  the  presentation  of  the  2016 
IEEE  G.  Fink  award  (York  et  al  2014).  We  have  also  made  an  important  discovery  of  how  fish  achieve 
an  optimal  reflective  silvery  camouflage  by  controlling  the  non-polarizing  properties  of  the  skin  (Jordan 
et  al  2012,  2013,  2014,  Roberts  et  al  2014).  We  have  extended  this  work  to  discover  the  greater 
importance  of  intensity  contrast  over  polarization  as  the  important  dimension  for  underwater  predators  to 
view  prey  (Johnsen  et  al  2016).  This  has  led  to  a  high-level  debate  in  the  magazine  Science  (Cronin  et  al 
2016). 

(4)  To  investigate  the  co-evolution  of  visual  systems,  visual  signals  and  camouflage  in  the  natural 
environment.  We  have  discovered  new  optical  physics  for  the  way  stomatopods  (and  comparatively 
their  great  rivals  the  cephalopods)  spectrally  control  the  polarization  and  visualisation  of  light  (Chiou  et 
al  2012,  Roberts  et  al  2012,  Talbot  et  al  2012,  How  et  al  2014,  Gagnon  et  al  2016).  We  also  discovered 
the  first  evidence  of  how  polarization  signals  evolved  in  stomatopods  and  how  these  signals  are  used.  We 


DISTRIBUTION  A.  Approved  for  public  release:  distribution  unlimited. 


have  also  extended  this  work  to  other  aquatic  species  including  cephalopods  and  other  crustaceans 
(Hanlon  et  al  2013,  Chung  et  al  2014,  How  2015)  and  with  other  comparisons  to  signalling  systems  and 
their  perception  across  all  animals  (Marshall  and  Cheney  2013,  Marshall  and  Arikawa  2014,  Cronin  et  al 
2014,  Kemp  et  al  2015,  Marshall  et  al  2015). 


Three  reviews  of  the  area  of  polarisation  vision  were  published  by  the  team  including  contributions  to  a 
new  book  on  the  subject  and  a  book  on  visual  ecology  as  a  whole  (Marshall  et  al  2014,  Marshall  et  al 
2014,  Cronin  et  al  2014). 
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Experiment: 


Methodological  descriptions  around  each  sub-section  are  included  in  the  Results  and  Discussion  section 

below.  They  include: 

a)  A  new  behavioural  paradigm  using  fro nt-polaroid- stripped  computer  monitors.  This  can  be  used  to 
assay  presence  /  absence  of  polarization  vision  and  acuity  of  polarization  vision  both  in  angle  of 
e- vector  discrimination  and  degree  or  %  polarization  (Fig.  2). 

b)  Standard  intracellular  electrophysiology  and  electroretinogram  (ERG)  using  extracellular  electrodes 
(Fig?). 

c)  A  new  (the  world’s  first)  underwater  and  terrestrial  polarization  camera.  This  has  been  used  to 
observe  and  characterize  polarized  light  and  polarization  signals  from  animals  in  the  field  (the  Great 
Barrier  Reef)  and  In  quantifying  polarized  light  field  for  GPS-free  geolocation  (Fig.  x). 

d)  Standard  and  3-view  electron  microscopy  to  characterize  nano-structure  of  photoreceptors  and  optical 
elements  involved  in  circular  polarsation  vision  (Fig.  x). 

e)  Standard  behavioural  paradigms  involving  two-way  choice  tests  with  reward  of  either  food  or 
safe-burrow.  Tests  were  made  against  colour,  linear,  eliptical  and  circular  polarisation  discrimination 
ability  (Fig.  4,5). 

f)  Old  (e.g.  Golgi)  and  new  (triple  fluorescent  stain  -  phalloidin,  a-tubulin,  DAPI)  neuroanatomical 
staining  and  characterization  techniques  to  characterize  sub-retinal  connections  and  information  flow 
through  to  the  brain.  (Figs.  3,7,8,9,10). 
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Results  and  Discussion: 


General  Significance  and  Outcomes  as  follows: 

The  outcomes  of  our  previous  four  years  of  work  include  over  70  publications,  many  in  the 
highest-impact  journals  (Science,  Current  Biology,  Nature  journals),  the  now  defining  text  book  for 
Visual  Ecology  (Cronin  and  Marshall),  two  patents  filed  (Gruev),  three  post  docs  have  become  new 
faculty  members  or  have  start  on  tenure  track  paths,  and  15  postgraduates  have  successfully  obtained 
their  PhDs.  These  outcomes  represent  evidence  for  the  impact  and  innovation  generated  from  this  grant. 

Our  work  has  influenced  areas  of  science  outside  our  own  immediate  fields.  These  include  areas  of 
primate  visual  processing  (Zaidi  2014)  the  field  of  cancer  diagnosis  (Charanya  2014),  and  early  diagnosis 
of  age-related  macular  pigment  degeneration,  the  leading  cause  of  blindness  in  the  western  world 
(Temple  2015). 

The  bio-inspired  development  of  the  world’s  first  real-time  underwater  (and  terrestrial)  polarization 
analysis  video  camera  has  forged  highly  successful  links  across  the  subject  areas  of  our  team  (two 
biologists,  one  physicist  and  one  engineer).  Evidence  and  recognition  of  this  successful  synergy  is 
demonstrated  by  the  presentation  in  2015  of  The  G.  Fink  award  by  the  IEEE  for  the  most  influential 
paper  published  in  the  Proceedings  of  the  IEEE  journal  (York  2014). 

Results  based  on  the  specific  questions  asked  in  the  Marshall  lab.  during  this  project  are  detailed 
below: 

(1)  We  aimed  to  learn  the  design  foundations  stomatopods  use  for  optimized,  rapid  processing  of 
multi-channel  information. 

Question.  What  is  the  sensitivity  of  polarization  vision  systems?  Using  new  behavioural  techniques, 
stomatopod  and  other  invertebrate  polarisation  sensitivities  were  described. 

In  2012  we  determined  that  stomatopods  have  surprisingly  coarse  spectral  and  polarisation  resolution, 
using  a  new  behavioural  technique.  This  involved  stripping  the  front  polaroid  off  computer  monitors  and 
producing  a  screen  and  stimuli  on  the  screen  that  could  only  be  viewed  by  animals  with  polarisation 
vision.  Conversely,  using  the  same  technique  we  demonstrated  that  polarisation  resolution  in  cuttlefish 
and  fiddler  crabs  was  relatively  fine  (Fig.  2). 

In  2013  we  aimed  to  develop  models  and  circuit-diagrams  that  account  for  this  difference  between 
stomatopods  and  other  animals,  test  other  stomatopod  species  and  other  non- stomatopod  species  for 
polarisation  resolution. 

This  objective  was  largely  achieved,  a  specific  model  for  polarisation  distance  (discrimination)  was 
developed  (How  and  Marshall  2014)  and  several  animals  tested  for  performance  (Fig.  3).  As  mentioned 
above,  interestingly,  as  also  reported  for  colour  vision,  stomatopods  under-perform  relative  to  other 
animals  in  the  tests  given.  The  How-Marshall  model  allows  neuroanatomical  results  to  be  combined  in  a 
model  that  estimates  polarisation  distance  or  contrast  between  two  objects  in  a  polarisation  space.  This  is 
similar  in  principle  to  chromatic  or  colour  space  and  colour  distances  or  contrasts.  Colour  vision  in 
stomatopods  has  been  studied  in  parallel  with  publications  in  Science,  Nature  and  Current  Biology 
resulting  -  see  next  specific  aim  for  description. 
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Fig.  2  Polarisation  test  paradigm  A-C  Left,  show  setup  with  cuttlefish.  The  looming  black  circle  can  only  be 
seen  by  animals  with  polarisation  vision  and  their  reaction  is  recorded.  Cuttlefish  blanch  or  change  body  colour 
as  shown  in  B  -  top  half  before,  bottom  half  after  stimulus.  This  can  be  quantified  using  pixel  brightness  and  is 
sensitive  down  to  one  degree  of  polarisation  difference.  Stomatopods  (Right)  also  react  to  similar  looming 
stimuli  by  retreating  or  showing  eye  movements.  However,  the  reaction  threshold  is  far  coarser  than  expected 
and  is  closer  to  15  degrees. 


Simple  neural  wiring  models  can 
Be  used  to  predict  how  polarisation 
Information  is  encoded  by  the  brain 


Polarisation  information 


Neuroanatomy  and  Computational  models  allow  estimates  of 
polarisation  contrast  signals 


Fluorescent 
labelling  of  eye 
and  brain  neurons 


Fig.  3  Neural  connectivity  and  modelled  polarisation  discrimination  system.  Fluorescent  labelling  of  stomatopod 
brain  structures  are  revealing  the  information  flow  to  the  central  processor.  This  enables  models  of  the  discrimination 
system  to  be  developed  where  the  polarisation  difference  between  object  and  background  is  estimated.  The 
How-Marshall  model  of  polarisation  distance  was  proposed  in  2013  (How  and  Marshall  2014). 
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In  2014/15  work  within  this  question  focussed  on  details  of  circular  polarisation  performance.  Having 
discovered  during  this  project  that  some  species  use  circular  or  elliptical  polarisation  instead  of  or  as  well 
as  hnear  polarisation,  the  behavioural  parameters  of  this  were  tested  in  a  different  apparatus  with  results 
indicating  that  this  form  of  light  was  used  in  resolving  aggressive  conflict  (Fig.  4.) 


Side  view: 


light  source 


dark-box 


Fig.  4  Stomatopods  fight  over  burrows  and  cavities  that  they  use  as  homes  on  the  reef  (a).  As  seen  in  (a)  and  (b),  they  often 
curl  into  a  defensive  posture  and  (b)  shows  areas  on  the  legs  and  body  of  Gonodactylus  falcatus  that  reflect  circular  polarised 
light.  Experimental  details  shown  in  (c)  are  from  recent  behavioural  tests  (Gagnon  et  al  2016)  showing  that  animals 
investigating  an  arena  avoid  cavities  containing  circular  polarisation,  presumably  as  these  burrows  are  assumed  occupied  by  a 
potential  competitor.  This  sort  of  information  transfer  using  cryptic  light  is  important  both  to  prevent  potentially  deadly 
conflict  between  animals  (for  example,  the  fight  seen  in  a  ended  in  the  death  of  one  of  the  stomatopods)  but  also  because 
circular  polarised  light  is  not  visible  to  other  marine  creature  as  far  as  we  know. 

One  of  the  areas  of  behavioural  work  this  leads  onto  in  the  next  proposal  is  to  examine  is  the  possible 
arms-race  between  stomatopods  and  cephalopods,  both  of  which  prey  on  each  other  and  both  of  which 
possess  linear  polarisation  vision  and  linear  polarisation  signals.  Only  stomatopods  (as  discovered 
through  this  project)  exhibit  circular  polarised  reflections  and  photoreceptors  capable  of 
seeing  this  light  modality.  Are  they  one  step  ahead  after  400  million  years? 
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(2)  We  tried  to  discover  how  information  is  processed  in  stomatopod  visual  systems.  At  higher 
levels,  we  examined  neural  interconnectivity  and  processing  principles  at  various  stages  of 
analysis  in  the  visual  system  both  anatomically  and  with  electrophysiological  approaches. 

Question.  How  do  the  photoreceptor  cells  of  mantis  shrimps  produce  responses  that  track  extremely 
transient  changes  in  stimulus  intensity? 

Extracellular  recordings  of  stomatopod  visual  responses  are  now  complete  and  tell  us  that  stomatopods 
have  a  relatively  rapid  response  time  compared  to  other  animals  (Chiou  et  al  Unpublished).  Given  the 
rapid  behaviours  they  engage  in,  including  ballistic  strikes  and  fighting  behaviour,  this  is  expected. 

Behavioural  results  from  both  colour  and  polarisation  processing  streams  (Question  1  above)  lends 
support  to  our  emerging  hypothesis  that  rapid  information  processing  should  be  possible,  despite  the 
multi-channel  nature  of  stomatopod  vision.  Essentially  redundant  information  is  discarded,  perhaps  in  a 
sparse-coding  scenario?  A  surprise  observation  was  that  in  fact  primates  (including  humans)  may  encode 
colour  information  at  higher  cortical  levels  as  stomatopods  do  at  retinal  level  (Zaidi  et  al  2013). 

Behavioural  colour  discrimination  tests  in  stomatopods  also  indicate  coarse  coding  of  colour  information 
(Fig.  5,  Thoen  et  al  2014). 
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Fig.  5  Spectral  discrimination  curves  (Ak/k).  Spectral  discrimination  curve  from  behavioural  testing  of  //.  trispinosa  (red 
dot-dash  line),  theoretically  modelled  spectral  discrimination  curve  (black  dot-dash  line)  compared  to  other  animals.  This 
demonstrates  that  stomatopods  encode  colour  at  a  low  bit-rate  and  possibly  do  not  use  the  same  neural  mechanisms  for  colour 
sense  as  other  animals. 

Resulting  from  both  polarisation  and  colour  behavioural  data  suggesting  stomatopods  encode  visual 
information  differently,  a  hypothetical  process  for  this  has  been  raised.  This  is  shown  in  Fig.  6.  Our  idea 
is  that  both  spectral  and  polarisation  information  is  viewed  as  an  instantaneous  pattern  of  excitation  - 
both  along  the  spectrum  and  around  the  polarisation  space  sphere  (Poincare).  This  pattern  is  read  out 
during  eye  scans  -  just  like  a  bar-code  scanner  at  a  store  -  and  enables  rapid  information  processing. 
Such  rapid  processing  of  information  from  con-specifics  and  other  competitors  may  be  a  necessary  part 
of  their  fast  and  violent  lifestyle. 
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Fig.  6.  Proposed  processing  mechanism  behind  stomatopod  colour  and  polarisation  vision,  a)  Idealised  spectral 
reflectance  from  stomatopod  body  parts  b)  Spectral  sensitivities  throughout  the  spectrum  divided  into  separate 
bins,  c)  and  d)  Excitation  patterns  of  each  spectral  sensitivity  when  looking  at  the  blue  c)  and  the  red  d) 
reflectance  spectra,  e)  Polarisation  system  in  stomatopods  with,  respectively,  linear  (90°  angles),  elliptical  (45° 
angles)  and  circular  (circles)  polarisation  sensitivities  binned  in  the  same  way  as  the  spectral  sensitivities. 


Question.  How  are  the  neural  circuits  constructed  to  provide  the  information  processing?  This  is  a  large 
question  requiring  many  years  work  but  good  progress  was  made  in  2013  -  2015.  PhD  students  Hanne 
Thoen  and  Rachel  Templin  have  begun  high  resolution  transmission  electron  microscopy  and  fluorescent 
staining  of  the  stomatopod  nervous  system  to  determine  information  pathways  arising  from  the  various 
visual  modalities.  This  forms  a  major  part  of  what  the  Marshall  lab  will  continue  with  in  2016-2020  if 
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successful  in  the  next  grant  application. 

There  are  excellent  results  from  both  Hanne  and  Rachel  from  the  last  four  years  work.  This  is  a  huge  task, 
however  Hanne  went  to  work  with  Nick  Strausfeld  in  Arizona  to  learn  his  world-leading  anatomical 
techniques. 

Discoveries  fall  into  three  categories  and  they  are  discussed  here  with  some  reference  to  planned  future 
work.  This  work  is  being  put  to  press  now  in  a  series  of  4  manuscripts  from  Hanne  Thoen’s  thesis. 

i)  Stomatopods  with  different  lifestyles  show  very  different  designs  in  neural  processing  units. 

The  hemielipsiod  bodies,  that  in  stomatopods  may  be  the  equivalent  of  the  memory-forming  mushroom 
bodies  in  insects,  appear  to  be  large  in  wandering  species  and  small  in  sedentary  species  (Figs.  7,9,10). 
Projections  from  the  olfactory  lobes,  which  are  the  primary  olfactory  centres  in  the  crustacean  brain,  are 
divided  into  two  branches,  each  terminating  in  the  hemiellipsoid  body.  An  interesting  aspect  is  the  fact 
that  in  some  insects  the  mushroom  bodies  have  been  found  to  process  visual  input,  either  as  part  of  a 
complete  modality  switch  or  by  topographical  division  of  the  different  modalities  into  separate  areas.  The 
distinct  projections  of  olfactory  neurons  into  the  outer  layers  of  the  hemiellipsoid  body  in  stomatopods 
suggests  that  the  inner  core  could  have  a  different  function,  perhaps  processing  visual  information? 
Hypotheses  arising  from  this  include:  i)  Stomatopod  hemiellipsoid  bodies  receive  and  encode  visual 
input,  ii)  Stomatopod  species  that  roam  freely  have  larger  hemiellipsoid  bodies  than  the  species  living  in 
burrows. 


Fig.7.  Representative  stomatopods  from  the  comparative  study  that  this  proposal  aims  to  expand  upon.  Left  -  the  animal. 
Centre  -  the  eye.  Right  -  fluorescent  immunolabel  of  eye-stalk  neuropils.  Note  the  relative  difference  in  size  of  the  various 
neural  integration  centres  such  as  the  hemielipsoid  bodies  (HB).  Scale  500um. 


We  plan  to  test  this  in  the  future  with  the  following  experiment.  In  simple  terms  stomatopods  across 
several  families  have  two  main  lifestyles;  a)  a  home  or  burrow-bound  existence  as  seen  in  Haptosquilla 
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trispinosa  for  example  or  b)  a  wandering  existence  as  seen  on  the  reef  flats  or  mud  plain  species  and 
around  the  reefs.  In  insects,  this  lifestyle  difference  presents  as  two  very  different  brain  morphologies, 
with  the  wanderers  having  larger  memory-modules  (the  mushroom  bodies)  as  might  be  expected. 
Stomatopods  also  show  differences  between  lifestyle  types,  however  in  their  case  it  appears  to  be  the 
hemielipsoid  body,  a  potential  mushroom  body  equivalent,  that  shows  marked  differences  between 
groups.  Working  closely  with  insect  and  invertebrate  brain  experts,  Nick  Starusfeld,  we  propose  to  not 
just  compare  brain  area  sizes  but  attempt  to  manipulate  them  through  developmental  constraints. 

The  stomatopod  experiment  will  involve  catching  the  different  stomatopod  types  (wanderers  and 
home-bound)  at  larval  stages  -  something  we  do  every  year  in  the  field  -  and  then  allowing  development 
of  each  type  in  relative  confinement  or  freedom.  The  prediction  is  that  wanderers  that  are  confined  to 
remain  home  for  several  weeks  will  not  develop  the  hemielipsoid  complexity  seen  in  individuals  that  are 
allowed  to  wander  in  larger  aquaria.  Comparison  of  the  neural  architecture  in  four  groups;  1  -  normal 
adult  wanderer,  2  -  normal  adult  home-bound,  3  -  developmental^  constrained  adult  wanderer  and  4  - 
developmental^  constrained  adult  home-bound  will  give  insight  into  how  stomatopods  from  different 
ecological  and  phylogenetic  backgrounds  process  information  and  the  plasticity  of  this  process  at  the 
neuronal  level. 

ii)  A  new  neural  processing  unit  (neuropil)  was  discovered  in  the  protocerebral  complex  of  some 
stomatopod  species  (Fig.  8,9,10).  For  the  moment  called  Thoen’s  Organ,  the  internal  anatomy  of  this 
structure  is  highly  glomerular  with  each  unit  passing  axons  through  the  optic  nerve.  Our  current  thinking 
is  that  each  glomerluar  unit  resolves  a  sub-category  of  visual  information  from  the  mid-band  and  other 
eye  regions.  Further  anatomical  description  and  close  electrophysiological  examination  of  this  new  brain 
region  are  planned  and  fortunately  this  lobe  is  situated  in  an  area  easy  to  record  from  with  intracellular 
electrodes.  Glomerular  complexes  in  flies  have  been  shown  to  receive  projections  directly  from  specific 
types  of  lobula  columnar  neurons  and  also  from  wide-field  neurons  in  the  lobula  receiving  input  from  the 
medulla.  These  glomeruli  were  shown  to  code  noisy  visual  primitives  from  single  lobula  output  neurons, 
thereby  providing  reliably  coded  information  from  converging  sensory  inputs.  In  Drosophila,  relays  from 
the  glomeruli  in  the  lateral  protocerebrum  projects  to  higher  centres  of  processing  such  as  the  central 
complex  through  the  dorsal  protocerebral  lobes.  But  they  also  convey  information  to  the  rest  of  the  body 
through  interactions  with  the  giant  descending  neurons,  which  project  to  the  thoracic  ganglia.  Although 
we  still  do  not  know  the  function  of  the  glomeruli  and  why  they  are  gathered  in  a  large  glomerular 
complex  in  the  stomatopod  protocerebrum,  it  is  likely  that  the  large  variety  of  shapes  and  sizes  of 
glomeruli  indicate  different  types  of  information  inputs,  possibly  from  different  types  of  visual 
modalities. 


Hypothesis:  Large-field  lobular  neurons  encode  spectral  and  polarisation  modalities  during  scanning 
eye  movement.  These  sub-categories  of  information  are  passed  to  Thoen’s  Organ  in  usable  packets  to  be 
sent  to  the  brain.  We  hope  to  test  this  in  the  new  work  proposed  for  2016-2020. 


Fig.  8  Details  of  Thoen's  Organ,  the  glomerular  body  that  forms  part  of  the  protocerebral  complex  in  gonodactyloid 
stomatopods  (see  Fig. 7).  Right  -  Three  dimensional  reconstruction  of  eye-stalk  structures  and  rotated  to  show  Thoen’s  Organ 
in  green.  Centre  -  Golgi  stain  of  nodular  structures  within  Thoen’s  Organ. 
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Fig.  9  Triple  immuno  stain  of  retina  and  underlying 
neuropils  in  Lysiosquilla  maculata  (Fig.  1).  Inset  below 
shows  eyes  in  life.  Note  relative  size  of  hemiellipsoid 
body  compared  to  squilloid  and  gonodactyloid  species  in 
Figs.  7,  10.  Box  encompasses  area  where  mid-band 
information  processing  is  separated  from  hemispheres. 


Fig.  10  Triple  immuno  stain  of  retina  and  underlying 
neuropils  in  Squilla  sp.  Inset  below  shows  eyes  in  life. 
Note  relative  size  of  hemiellipsoid  body  compared  to 
lysiosquilloid  and  gonodactyloid  species  in  Figs.  7,  10. 
Arrow  indicates  area  where  mid-band  information 
processing  is  separated  from  hemispheres. 
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iii)  Insects  are  stomatopods  of  the  air.  Very  new  comparative  results  suggest  that  stomatopods  may  be 
the  progenitors  of  dicondylic  (most)  insects.  This  surprising  possibility  is  the  brain-child  (pardon  the  pun) 
of  Nick  Strausfeld  and  Hanne  Thoen  and  comes  from  a  comparison  of  the  central  complex  of  a  number 
of  insects  and  crustaceans.  In  conjunction  with  Nick  Strausfeld,  one  aim  here  is  to  test  this  hypothesis 
against  the  other  possibility  that  stomatopod  brain  structure  is  highly  converged  with  insects,  potentially 
unlikely  given  their  different  lifestyles,  or  is  it?  This  has  been  proposed  in  new  grant  application.  Very 
recent  results  have  been  published  (Thoen  et  al  2017)  and  have  led  to  a  new  hypothesis  on  the  evolution 
and  phytogeny  of  all  arthropods  which  we  are  trying  to  get  published  in  Nature  now  (Wolff  et  al  2017) 

Hypothesis:  Central  complex  (CX)  in  arthropods  fall  into  two  gross  structural  categories.  The  CX  in 
stomatopods  is  more  like  that  of  insects  than  other  crustaceans  and  further  comparative  investigation  of 
this  over  several  insect  and  crustacean  lineages  will  provide  evidence  for  one  or  the  other  hypothesis. 


(3)  To  attempted  to  discover  more  of  the  natural  complexity  of  visual  scenes  the  stomatopod 
imaging  system  has  evolved  to  see. 

Question,  development  1.  Building  a  real-time  polarisation  camera  for  polarisation  visual  information 
analysis  underwater.  Development  of  real-time  polarisation  cameras  with  collaborator  Gruev  has  been 
successful  (Fig.  11)  and  field  tests  on  The  Great  Barrier  Reef.  Several  new  applications  of  this  camera 
have  resulted  including  the  medical  avenue  of  early  cancer  detection.  (York  et  al  2014). 

Question,  development  2.  Camouflage  and  communication  underwater  using  polarized  light.  Imaging  of 
animals  in  their  natural  habitat  is  continuing  and  has  resulted  in  a  series  of  papers  and  chapters  published 
and  due  this  year  this  year.  Silvery  camouflage  was  examined  in  2012  with  a  Nature  publication  resulting 
from  this  work.  In  2013,  further  camouflage  and  communication  in  polarisation  space  will  be  examined 
in  both  underwater  and  terrestrial  habitats. 


Fig.  12  is  a  sample  of  the  many  animals  and  objects  examined.  These  include  underwater  polarisation 
targets  for  calibration  of  the  system  and  animals  that  use  polarisation  as  a  source  of  information.  In  the 
case  of  the  stomatopod  (third  from  bottom)  this  information  is  used  in  mate  selection.  The  cuttlefish, 
immediately  below  the  stomatopod,  also  appears  to  use  polarisation  information  for  communication.  As 
these  animals  lack  colour  vision  but  have  very  acute  polarisation  vision  (Temple  et  al  2012)  we 
hypothesise  that  they  use  polarisation  in  the  same  way  as  other  animals  use  colour.  Advantages  of  this 
new  mode  of  visual  communication  are  being  investigated  now  and  form  a  major  component  of  the 
current  grant  proposal. 
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Fig.  1 1  First  underwater 
real-time  polarisation  camera. 
Left  -  the  camera,  computer 
and  battery  are  fitted  and 
controlled  in  an  underwater 
housing.  Right  -  the 
innovative  core  of  the  system 
is  a  nanofabricated  pixel  mask 
including  vertical,  horizontal 
and  45  degree  polarisation 
elements  (in  place  of  RGB). 
As  implied  here,  a  circular 
polarisation  camera  is  in 
design  now. 
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Fig.  12  Results  from  underwater  polarising  camera.  False  colour  images  of  various  animals  and  other  objects  showing  strong 
degree  of  polarization  contrast  against  background  (middle  column)  and  intrinsic  polarization  information  in  stomatopods, 
cuttlefish,  reflections  and  underwater  polarization  targets. 
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(4)  We  investigated  the  co-evolution  of  visual  systems,  visual  signals  and  camouflage  in  the 
natural  environment. 

This  work  was  largely  coordinated  in  the  Cronin  and  Roberts  labs,  however  Marshall  lab.  PhD  student 
Rachel  Templin  has  spearheaded  a  successful  project  examining  the  co-evolution  of  circular  /  elliptical 
polarisation  signals  and  the  R8  photoreceptor  cells  responsible  for  facilitating  this  form  of  vision,  and  its 
relative  elipticity,  in  different  stomatopod  species.  We  set  out  with  two  specific  questions: 

Question  1.  Are  the  polarization  signals  from  different  stomatopod  species  matched  to  species-specific 
elliptical  polarization  vision  capability?  New  anatomical  studies  will  be  initiated  this  year  to  examine  this 
question. 

Question  2.  As  well  as  between  species  R8  differences  (and  therefore  elliptical  information  tuning),  very 
recent  results  suggest  within- species  differences  in  some  species.  Is  this  the  outcome  of  an  ‘arms  race’  in 
which  there  are  generalist  species,  able  to  interpret  the  messages  of  prey,  break  prey  camouflage,  and 
‘read  the  mail’  of  the  unsuspecting  messengers? 


Fig.  13.  Left  -  electron  micrograph  detail  of  the  R8  cell  showing  crystalline  arrangement  of  microvilli.  The  anatomical 
parameters  of  this  structure,  tube  or  microvillar  width  and  the  total  length  of  the  R8  cell,  give  different  stomatopod  species 
specific  optical  retardation  characteristics  as  plotted  -  Right  -  against  a  theoretically  perfect  14  wavelength  retardation.  lA 
wavelength  retardation  results  in  circular  polarisation  vision  (Marshall  et  al  et  al  2014)  as  is  the  case  here  for  Lysiosquilla 
maculata  (Fig.  1). 

Rachel’s  PhD  which  is  beginning  to  be  published  this  year  examines  different  stomatopod  species  and 
their  R8  photoreceptors  (Fig.  13)  as  well  as  their  signalling  characteristics  and  behaviours  (Fig.  7) 
relative  to  circular  polarisation.  Our  current  hypothesis  is  different  stomatopods  use  different  elipticities 
and  R8  cell  characteristics  to  tune  polarisation  vision  to  specific  signals,  in  much  the  same  way  as  colour 
vision  may  be  tuned  to  specific  colours. 
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